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ABSTRACT 


The  U.  S.  Navy  currently  has  no  means  to  conduct  sea  mine  reconnaissance  with 
assets  that  are  organic  to  Aircraft  Carrier  Battle  Groups  or  Amphibious  Ready  Groups. 
Magic  Lantern  is  an  Airborne  Laser  Mine  Detection  System  (ALMDS)  under  development, 
that  is  designed  to  search  for  floating  and  shallow  moored  mines  using  a  helicopter- 
mounted  laser-optic  sensor.  It  is  the  only  ALMDS  operationally  tested  by  the  Navy  to  date. 

This  thesis  develops  a  Monte  Carlo  simulation  model  called  ML-Recon,  which  is 
intended  for  use  as  a  tool  to  plan  mine  reconnaissance  searches  using  the  Magic  Lantern 
system.  By  entering  fundamental  initial  plamiing  information,  the  user  can  determine  the 
number  of  uniformly-spaced  tracks  to  fly  with  a  Magic  Lantern-equipped  helicopter  to 
achieve  a  certain  level  of  assurance  that  the  area  contains  no  floating  or  shallow  moored 
mines.  By  employing  Monte  Carlo  methods,  ML-Recon  models  the  three  primary 
stochastic  processes  that  take  place  during  a  typical  search;  the  location  of  the  mines,  the 
cross-track  error  of  the  helicopter,  and  the  detection/non-detection  process  of  the  sensor. 

By  running  ML-Recon,  the  user  is  given  performance  statistics  for  many 
replications  of  the  search  plan  that  he  chooses.  This  approach  is  unique  in  that  it  provides 
the  user  with  information  indicating  how  much  worse  than  the  mean  performance  his  plan 
may  perform .  ML-Recon  also  gives  the  user  an  opportunity  to  view  an  animation  of  his 
plan,  which  he  can  use  to  look  for  tendencies  in  the  plan  to  contain  holes,  or  holidays. 
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EXECUTIVE  SUMMARY 


The  U.S.  Navy  currently  has  no  mine-hunting  asset  organic  to  Aircraft  Carrier 
Battle  Groups  (CVBGs)  or  Amphibious  Ready  Groups  (ARGs).  Mine  Countermeasures 
(MCM)  planners  agree  that,  to  conduct  adequate  sea  mine  reconnaissance,  a  combination  of 
systems,  each  specializing  in  searching  a  different  depth  layer,  is  required  to  cany  out 
adequate  mine  reconnaissance  for  a  battle  group  transiting  through  potentially  mined 
waters.  A  number  of  systems  are  currently  under  development,  one  of  which  is  the 
Airborne  Laser  Mine  Detection  System  (ALMDS). 

ALMDS  is  a  laser-optic  mine-hunting  sensor  mounted  on  a  helicopter  that  would 
give  CVBGs  the  capability  to  search  for  floating  and  shallow  moored  mines  if  it  were 
adapted  to  the  Navy’s  current  Light  Airborne  Multi-Purpose  System  (LAMPS)  helicopter, 
the  SH-60.  Of  the  ALMDS  systems  being  considered.  Magic  Lantern,  a  system  developed 
by  Kaman  Corporation,  is  the  only  one  that  has  undergone  operational  tests  by  the  Navy. 

The  simulation  model  developed  in  this  thesis,  ML-Recon,  was  built  as  an 
interactive  computer  planning  tool  for  the  employment  of  Magic  Lantern.  It  models  the 
stochastic  processes  that  take  place  during  a  typical  search,  and  provides  the  planner  with  a 
realistic  estimate  of  the  expected  performance  of  his  plan.  The  following  steps  summarize 
the  operation  of  this  planning  tool. 

1)  The  user  is  prompted  to  input  the  size  of  the  channel  intended  for  the  search,  the 
distance  to  the  channel  from  the  helicopter’s  base  ship,  the  number  of  mines  expected  in  the 
channel,  and  the  number  of  replications  of  the  search  plan  the  user  would  like  to  run. 

2)  Since  the  sensor’s  probability  of  detection  capability  varies  with  depth,  a  set  of 
four  graphs  is  presented  to  the  user,  each  of  which  plots  Percent  Coverage  vs.  Number  pf 
Tracks  for  a  different  depth  layer.  The  first  depth  layer  begins  at  the  surface,  and  each 
subsequent  one  represents  a  deeper  volume  than  its  predecessor.  From  these  graphs  the 
user  can  estimate  the  number  of  tracks  to  run  to  achieve  his  desired  coverage.  The  user 
would  be  expected  to  determine  the  number  of  tracks  from  the  worst  performing  depth 
layer.  Also  provided  is  a  graph  with  an  estimate  for  the  time  required  to  complete  the 
search.  After  examining  the  planning  graphs,  the  user  enters  the  number  of  tracks  desired 
for  his  search  plan. 

3)  The  replications  of  the  search  plan  are  run,  and  performance  statistics  for  each 
completed  search  are  recorded.  During  one  replication,  the  helicopter  transits  from  the  base 
ship  to  the  channel  and  searches  for  randomly  located  mines  by  flying  the  specified  number 
of  tracks.  The  tracks  are  spaced  uniformly.  However,  a  random  cross-track  error  is 
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imparted  on  the  helicopter’s  actual  flight  path.  As  the  helicopter  is  flying,  the  laser  is 
independently  emitting  pulses  into  the  water.  Fuel  consumption  is  monitored  throughout 
the  search.  During  each  track,  if  a  mine  is  determined  to  be  within  the  laser’s  sweep  width, 
its  depth  and  lateral  range  from  the  helicopter  are  checked,  and  the  appropriate  probability 
of  detection  is  applied  in  an  attempt  to  detect  the  mine.  There  are  twenty  different 
probabilities  of  detection  used  in  the  model. 

4)  When  all  replications  are  complete,  the  user  is  shown  four  performance  graphs, 
one  for  each  depth  layer.  The  graphs  plot  the  individual  performance  (mines  detected)  of 
each  replication  and  the  running  average  for  all  the  replications. 

5)  The  user  is  offered  the  choice  of  viewing  an  animated  version  of  the  search, 
repeating  the  replications  with  a  different  number  of  tracks,  or  quitting  the  program. 

Unique  features  of  ML-Recon,  not  present  in  any  operational  MCM  planning  tool, 
are  the  randomly  modeled  stochastic  processes  and  the  animation.  The  stochastic  processes 
include  1)  the  random  positions  of  the  mines  in  the  channel  for  each  new  replication  of  the 
search  plan,  2)  the  random  navigational  error  applied  to  the  helicopter’s  flight  path,  and 
3)  the  sensor’s  probabilistic  prosecution  of  each  individual  mine  it  encounters. 

The  value  of  the  animation  is  the  visual  representation  of  the  helicopter’s  errant 
flight  path  and  the  associated  holes,  or  holidays,  in  the  search  plan.  Since  the  laser  trails 
remain  on  the  screen,  after  the  search  is  complete  the  planner  can  determine  the  number  of 
mines  missed  because  of  poor  sensor  performance  and  those  missed  because  of  holidays. 

For  model  verification,  ML-Recon  compares  favorably  with  an  accredited  analytical 
MCM  planning  program.  Uniform  Coverage  Plan  (UCPLN).  Yet,  by  running  several 
replications  and  employing  random  numbers  to  model  the  stochastic  processes  of  the 
search,  ML-Recon  provides  more  information  to  the  planner  about  the  spread  of 
possibilities  in  the  performance  of  the  search  plan. 


I.  INTRODUCTION 


A.  BACKGROUND 

The  U.S.  Navy  currently  has  no  mine-hunting  asset  organic  to  Aircraft  Carrier 
Battle  Groups  (CVBGs)  or  Amphibious  Ready  Groups  (ARGs).  Once  a  mine  threat  has 
been  identified  in  a  particular  theater  of  interest.  Mine  Countermeasures  (MCM)  ships  and 
helicopter  squadrons  are  deployed  to  the  area.  Their  arrival  may  take  days  or  longer, 
depending  on  their  transportation  requirements  and  the  availability  of  an  in-theater  logistics 
support  base.  In  the  meantime,  battle  groups  and  resupply  ships  in  theater  are  at  risk 
waiting  for  the  arrival  of  the  MCM  assets. 

A  concept  of  operations  document  regarding  sea  mine  reconnaissance  and  organic 
battle  group  self-protection,  written  by  the  Underwater  MCM  Office  of  the  Chief  of  Naval 
Operations  [Ref.  1],  states  that  currently  there  is  no  single  method  for  adequately 
conducting  sea  mine  reconnaissance.  No  individual  mine-hunting  system  can  effectively 
search  for  mines  from  the  surface  down  to  the  bottom,  and  below  the  seabed  for  buried 
mines.  Therefore,  a  suite  of  sensors,  each  one  focusing  on  a  different  depth  layer,  is 
required  in  a  battle  group  to  ensure  adequate  reconnaissance  in  the  event  that  mines  are 
suspected. 

Among  the  sensors  that  would  be  included  in  this  combination  is  an  Airborne  Laser 
Mine  Detection  System  (ALMDS).  An  ALMDS  is  a  helicopter-mounted  electro-optic 
sensor  designed  to  search  for  floating  and  shallow  moored  mines  by  emitting  laser  energy 
into  the  water  and  recording  reflected  energy  from  mine-shaped  objects.  The  sensor 
contains  a  classification  algorithm  that  determines  whether  a  reflected  signal  is  from  a  mine 
or  not. 

Magic  Lantern,  developed  by  Kaman  Corporation,  is  the  only  ALMDS  that  has 
been  operationally  tested  by  the  Navy.  Two  Magic  Lantern  sensors  are  presently  under 
contract  by  the  Navy,  and  others  are  being  considered  [Ref.  2].  Future  systems  would  be 
mounted  on  Light  Airborne  Multi-Purpose  System  (LAMPS)  helicopters,  which  are  an 
integral  part  of  all  CVBGs  and  conceivably  could  be  deployed  with  ARGs. 

In  August,  1994,  the  Airborne  Mine  Warfare  Systems  Program  Office  at  the  Naval 
Sea  Systems  Command  sponsored  operational  tests  of  Magic  Lantern  in  Fort  Walton 
Beach,  Florida  [Ref.  3].  Data  were  collected  during  these  tests  by  METRON,  an 
independent  scientific  consulting  firm,  in  conjunction  with  the  Coastal  Systems  Station  in 
Panama  City,  Florida.  The  data  collected  were  analyzed  to  determine  the  sensor’s 
capability  to  detect  mines,  a  characteristic  described  as  probability  of  detection  (P^).  The 
nature  of  modem  MCM  required  that  precise  navigational  data  for  the  helicopter  also  be 
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collected  during  each  of  the  mine-hunting  sorties.  These  data  reflect  a  navigational  error 
between  the  intended  track  and  the  actual  flight  pattern  of  the  helicopter. 

At  present,  ML-Recon,  the  simulation  described  in  this  thesis,  is  the  only 
performance  model  designed  specifically  for  the  Magic  Lantern  system.  In  addition,  it  is 
the  only  MCM  planning  aid  that  models  stochastic  processes  with  the  use  of  random 
numbers.  The  result  is  that  the  information  generated  by  ML-Recon  gives  the  planner  a 
more  thorough  understanding  of  the  potential  outcome  of  his  search  plan. 

B.  PROBLEM  STATEMENT 

The  MCM  planner  or  Battle  Group  Commander  approaching  potentially  mined 
waters  will  have  a  Magic  Lantern-equipped  helicopter  among  a  combination  of  mine¬ 
hunting  sensors  at  his  disposal.  Although  this  asset  will  not  neutralize  or  clear  mines,  it 
can  provide  the  planner  with  critical  reconnaissance  information  about  the  shallowest  water 
layer  of  an  intended  transit  route.  If  mines  are  detected,  an  alternate  route  can  be  chosen. 
If  mines  are  detected,  and  the  search  plan  is  adequately  thorough,  the  shallowest  layer 
can  be  considered  sanitized,  or  mine-free. 

Depending  on  the  size  of  the  channel,  the  search  may  require  a  very  carefully 
chosen  plan.  A  search  plan  that  is  too  extensive  may  not  be  completed  before  aircrew  rest 
or  refueling  and  scheduled  helicopter  maintenance  are  required.  A  plan  that  is  too  sparse 
may  miss  mines  due  to  holes  in  the  search.  Knowing  the  following  information: 

•  channel  size; 

•  percent  sanitization  desired; 

•  detection  capability  of  the  sensor; 

•  helicopter  and  aircrew  logistic  considerations; 
the  MCM  planner  needs  to  determine: 

•  the  number  of  tracks  to  fly  and  their  inter-track  spacing  to  complete  a 
reconnaissance  search  that  will  achieve  the  sanitization  percentage  desired,  and; 

•  the  time  required  to  complete  the  search. 

C.  APPROACH 

The  approach  to  solving  the  problem  is  to  analyze  the  Magic  Lantern  system  and  the 
data  collected  during  operational  tests,  and  to  construct  a  Monte  Carlo  simulation  model, 
ML-Recon,  that  can  be  used  as  a  search  planning  tool. 
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II.  THE  SYSTEM 


A.  OVERVIEW 

The  Magic  Lantern  system  consists  of  two  main  parts:  the  laser-optic  sensor  and  the 
helicopter  upon  which  the  sensor  is  mounted.  The  helicopter  used  in  operational  tests  thus 
far  has  been  the  SH-2  Seasprite,  also  manufactured  by  Kaman  Corporation.  Since  the 
SH-2  has  been  superseded  by  the  SH-60  Seahawk  as  the  battle  group  LAMPS  helicopter, 
the  sensor  will  ultimately  be  used  on  the  newer  airframe  [Ref.  2].  Adjustments  to  ML- 
Recon  in  order  to  conform  to  system  updates  are  discussed  later  in  the  model  description. 

B.  THE  HELICOPTER 

LAMPS  helicopters  are  embarked  on  most  cruisers,  destroyers  and  frigates  in  a 
CVBG.  These  ships  carry  one  or  two  helicopters  and  two  or  three  crews,  respectively 
[Ref.  5].  Refueling,  as  well  as  crew  changes,  can  be  conducted  while  the  helicopter  is  on 
deck  and  still  running.  However,  scheduled  periodic  maintenance  prevents  around-the- 
clock  operation  of  a  single  helicopter. 

For  the  purposes  of  this  study  a  helicopter  travels  at  two  speeds:  transit  speed  and 
search  speed.  As  the  names  imply,  transit  speed  refers  to  the  velocity  of  the  helicopter  as  it 
is  in  transit  between  the  channel  and  its  base  ship.  Search  speed  is  the  speed  at  which  it 
conducts  the  search.  Transit  speed  is  greater  than  search  speed.  As  helicopter  speed  is 
more_  a  function  of  blade  pitch  than  rotor  speed,  the  ratio  of  time  spent  at  transit  speed  to 
that  spent  at  search  speed  does  not  significantly  affect  the  frequency  of  refueling. 

C.  THE  SENSOR 

Aspects  of  the  sensor  that  are  pertinent  to  the  analysis  are  the  shape  and  structure  of 
the  search  volume  and  the  associated  characteristics.  Although  the  sensor  is  mounted 
outside  of  the  aircraft  on  the  starboard  side,  the  offset  can  be  pre-programmed  into  a  Global 
Positioning  System  (GPS)  navigation  unit,  so  the  sensor  is  assumed  to  be  at  the  centerline 
of  the  helicopter. 

Although  an  optimum  airspeed  for  the  sensor’s  performance  does  exist  [Ref.  3],  the 
operation  of  the  sensor  is  separate  from  the  helicopter  and,  thus,  independent  of  its  speed 
and  direction.  The  sensor  unit  emits  pulses  of  laser  energy  into  the  water  column  directly 
below  the  helicopter  at  a  constant  rate.  Laser  energy  is  reflected  from  items  in  the  water 
column  and  received  through  cameras  inside  the  sensor  unit.  Each  camera’s  shutter  is  set 
to  be  open  during  a  distinct  time  interval,  delayed  from  the  instant  the  pulse  is  emitted. 
Therefore,  each  camera  captures  reflected  energy  from  a  different  depth  zone  in  the  water 
column,  corresponding  to  the  time  that  the  shutter  is  open.  For  example,  reflections  from 
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the  shallowest  depths  are  received  by  the  camera  that  opens  and  closes  first.  Once  the 
reflected  energy  returns  to  the  sensor,  contacts  are  classified  by  a  detection  algorithm, 
assigned  a  location,  and  recorded. 

The  sweep  width  of  the  sensor  is  divided  into  several  sections,  as  shown  in 
Figure  1 .  Laser  pulses  are  emitted  at  a  constant  rate.  One  cycle  of  these  pulses  begins 


at  the  left  edge  and  moves  through  the  search  width  to  the  right  edge,  then  begins  again  on 
the  left  [Ref.  3].  Each  pulse  penetrates  the  water  so  that,  at  the  completion  of  one  sweep, 
the  sensor  has  searched  a  three-dimensional  space,  or  a  searchvolume. 

The  Pj  data  were  collected  in  such  a  way  that  the  search  volume  is  divided  into  five 
lateral  ranges  and  four  depth  zones  [Ref.  3],  for  a  total  of  twenty  subvolumes.  Figure  2 
shows  the  cross-section  of  the  search  volume  structure  (as  though  the  helicopter  were 
flying  away  from  the  reader)  as  it  is  described  by  the  data.  The  length,  width,  and  depth 
dimensions  of  the  search  volume  are  also  depicted.  The  surface  of  the  water  is  represented 

by  the  top  line  of  the  structure.  The  arrow  (.][)  in  the  figure  shows  the  nadir  position,  or,  the 

point  directly  below  the  helicopter.  „  in  the  figure  represents  the  P^  in  the  shallowest 
depth  layer  and  the  left-most  lateral  range  bin. 
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Figure  2.  Search  Volume  Structure 


The  structure  of  the  sensor’s  search  volume  reveals  that  a  mine’s  depth  and  lateral 
range  have  an  impact  on  whether  it  is  detected  or  not.  The  fact  that  the  sensor’s  capability 
varies  in  two  dimensions  (depth  and  lateral  range)  makes  it  unique  among  mine-hunting 
systems,  and,  hence,  not  well-represented  by  current  planning  models  such  as  the  Uniform 
Coverage  Plan  or  the  Non-Uniform  Coverage  Evaluation  program,  as  these  programs 
assume  a  single  P ^  value  for  the  sensor  used  in  the  search  [Ref.  6] . 

At  present,  this  sensor  has  no  rmnc-sweeping  or  clearing  capability.  The  laser  does 
not  detonate  mines  intentionally  or  otherwise.  Mines  in  the  water  that  have  been  flown 
over  and  detected  do  not  affect  the  helicopter’s  flight  in  any  way.  The  sensor  continuously 
emits  pulses  throughout  the  search,  and  the  helicopter  maintains  a  constant  speed, 
regardless  of  the  number  of  mines  below  or  whether  or  not  they  are  detected.  The 
positions  of  mines  that  are  detected  are  stored  for  future  analysis  by  planners  once  the 
search  is  complete. 
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III.  THE  MODEL 


A.  OVERVIEW 

ML-Recon  is  designed  to  simulate  a  helicopter  with  a  Magic  Lantern  sensor 
executing  a  specific  mine-hunting  plan.  The  user  is  prompted  to  enter  the  size  of  the 
channel  to  be  searched,  the  distance  from  the  helicopter’s  base  ship  to  the  channel,  the 
number  of  mines  to  plant  in  the  channel,  and  the  number  of  replications  of  the  search  he 
would  like  to  run.  Prior  to  selecting  the  number  of  tracks,  he  is  given  an  analytical  estimate 
of  the  approximate  coverage  he  will  achieve,  depending  on  the  number  of  tracks  run.  The 
helicopter  then  transits  to  the  channel  and  executes  the  search  plan.  The  sensor 
continuously  emits  laser  pulses  while  the  helicopter  flies  the  specified  number  of  tracks.  At 
the  conclusion  of  each  search  replication,  detection  results  are  recorded.  When  all 
replications  have  been  completed,  summary  graphs  showing  the  overall  results  are  shown 
on  the  screen,  and  the  user  is  offered  the  opportunity  to  see  an  animated  version  of  the 
search. 

ML-Recon  is  written  in  MODSIM  II,  an  object-oriented,  modular  simulation 
programming  language.  Therefore,  as  aspects  of  the  Magic  Lantern  system  improve  with 
more  advanced  technology,  changes  can  easily  be  incorporated  into  the  appropriate  module 
of  ML-Recon.  Many  of  the  parameters  of  the  system,  such  as  search  speed,  depth 
capability,  and  the  various  values  of  the  sensor,  are  simply  entered  as  constants  in  a 
calculating  module  of  the  program.  They  can  be  quickly  updated  with  system 
improvements.  Navigational  error  is  modeled  using  a  normal  random  variable  with  a 
specific  mean  and  variance.  If  a  different  helicopter  is  determined  to  be  equipped  with 
Magic  Lantern  (a  very  likely  possibility),  ML-Recon  need  not  be  rewritten.  Only  the  new 
helicopter’s  navigational  error  parameters  and  transit  speed  capability  are  required. 

In  this  thesis,  the  term  coverage  describes  a  percentage  of  water  volume  (a  chaimel 
with  length,  width  and  depth)  that  has  been  sanitized.  If  a  channel  is  95%  covered  then,  of 
all  the  mines  in  that  channel,  95%  of  them  are  detected.  From  another  perspective,  if  one 
mine  exists  in  the  channel,  the  probability  that  the  search  detects  it  is  0.95. 

B.  ANALYTICAL  APPROXIMATION 

The  value  of  ML-Recon  is  in  the  stochastic  processes  that  it  models.  However, 
since  a  large  number  of  replications  of  the  search  may  be  rather  time-consuming,  the 
planner  is  provided  with  a  graph  that  assists  him  in  his  initial  choice  for  the  number  of 
tracks  to  run.  The  graph  plots  the  expected  coverage,  p,  against  the  number  of  tracks 
flown,  n  (see  Figure  3).  Using  the  graph  as  a  guide,  the  planner  selects  a  close  estimate  of 


7 


the  number  of  tracks  that  should  be  flown  to  achieve  his  desired  coverage.  To  begin  ML- 
Recon  without  any  idea  of  the  number  of  tracks  to  run  would  likely  result  in  his  choosing 
several  bracketing  trials  to  arrive  at  the  ultimate  number  of  tracks.  To  derive  a  formula  for 


Estimated  Coverage  vs.  Number  of  Tracks 


Figures.  Analytical  Estimation  for  Coverage 

p,  assume  that  the  search  width  of  the  sensor  is  significantly  more  narrow  than  the  width 
of  the  channel  being  searched.  Let  be  the  sweep  width  of  the  sensor,  which  is  the 
product  of  the  overall  search  width  and  the  P^,  and  let  a  be  the  width  of  the  channel.  For 
independent  random  tracks  in  a  channel  [Ref.  7],  the  probability  that  a  given  point  is 
detected  after  n  tracks  is: 

p  =  1  -  (  1  -  Wa/a)" 

The  parametera  must  be  entered  into  ML-Recon  by  the  planner.  He  will  generally 
base  this  number  on  the  navigational  capabilities  of  the  ship(s)  that  will  ultimately  transit  the 
channel  once  it  has  been  deemed  safe.  The  guidance  in  reference  [6]  for  determining 
channel  width  is  to  use  at  least  six  times  the  standard  deviation  to  navigational  error  of  the 
transiting  ships.  The  value  for  w »  is  calculated  by  multiplying  the  overall  search  width  of 
the  sensor  by  its  aggregate  Pj  in  each  depth  zone.  Once  the  replications  of  the  actual  search 
begin,  ML-Recon  uses  five  distinct  probability  of  detection  values  across  the  lateral  range 
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of  the  sensor  (see  Section  C  of  this  chapter)  for  each  separate  depth  zone.  The  graph 
shows  the  probability  that  a  given  point  in  the  channel  is  effectively  searched  for  one 
through  one  hundred  tracks.  The  shape  of  the  graph  indicates  that  there  is  a  point  at  which 
adding  more  tracks  achieves  negligible  improvement  in  coverage.  Since  the  Magic  Lantern 
system,  as  described,  divides  the  water  column  into  four  depth  zones  and  has  a  different 
aggregate  Pj  for  each,  four  graphs  are  provided  to  the  user,  one  for  each  depth  zone.  He 
would  be  expected  to  select  the  number  of  tracks  required  from  the  graph  that  shows  the 
depth  zone  where  the  sensor’s  capability  is  worst. 

One  additional  graph  is  provided  before  ML-Recon  begins.  This  graph  plots  search 
time  against  the  number  of  tracks  flown  (Figure  4),  since  time  may  be  of  critical  interest  to 
the  planner.  The  graph  shows  him  approximately  how  long  his  search  will  take.  This  is  a 


Search  Time  vs.  Number  of  Tracks 


Number  of  Tracks 
Figure  4.  Estimated  Search  Time 

piece-wise  linear  representation  of  the  total  distance  traveled  at  a  predetermined  speed. 
Search  time  includes  the  time  required  to  turn  between  tracks  and  the  transit  time  to  and 
from  the  area.  The  roimd-trip  transit  time  can  be  seen  on  the  y-axis  of  the  graph  where  the 
number  of  tracks  is  zero.  The  jumps  in  search  time  occur  at  points  where  refueling  is 
required.  For  searches  of  veiy  long  transit  routes  (channels),  the  planner  may  decide  to 
break  up  the  search  into  a  munber  of  shorter  sections,  each  of  which  would  be  searched 
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separately.  Figure  5  shows  how  a  long  channel  may  be  divided  into  three  shorter 
chaimels.  By  viewing  the  relationship  between  search  time  and  the  number  of  tracks 


Total  Channel  Length  - ► 


Shorter 

< - Channel - » 

Section 


Figure  5.  Long  Channel  Divided  Into  Three  Shorter  Sections 

flown,  he  can  modify  the  length  of  the  channel  sections  so  that  a  section  is  searched 
completely  before  the  helicopter  must  return  to  its  host  ship  for  fuel. 

C.  STOCHASTIC  PROCESSES  IN  ML-RECON 

Three  specific  random  phenomena  are  modeled.  The  first  is  the  random  location  of 
mines  in  the  channel.  The  second  is  the  navigational  error  of  the  helicopter’s  flight  path. 
The  third  is  the  sensor’s  prosecution  of  potential  mine  contacts.  The  two  latter  processes 
are  modeled  on  the  data  that  were  collected  during  actual  tests  of  the  Magic  Lantern  system. 

1.  The  Mined  Channel 

The  channel  is  sized  in  three  dimensions.  The  x  and  y  dimensions  are  taken  from 
the  length  and  width  entries  specified  by  the  planner  at  the  beginning  of  the  program.  The  z 
dimension,  depth,  is  the  maximum  depth  to  which  the  sensor  is  capable  of  detecting  mines. 
This  dimension  is  divided  into  four  layers  to  represent  the  four  depth  zones  searched  by  the 
sensor. 

The  user  is  prompted  to  enter  an  estimate  for  the  number  of  mines  in  the  channel. 
This  is  done  for  two  reasons.  First,  some  positive  number  of  mines  in  the  channel  is 
necessary  in  order  to  quantitatively  assess  the  effectiveness  of  the  user’s  proposed  search 
plan.  Secondly,  although  it  is  unlikely  that  the  user  will  know  the  exact  number  of  mines 
that  are  in  the  channel,  he  may  have  some  idea  based  on  intelligence  information  or  on 
previously  observed  mine  densities  in  the  area.  In  such  cases,  he  should  have  the  ability  to 
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apply  that  information  rather  than  have  the  number  of  mines  unalterably  fixed  for  statistical 
convenience  by  a  programmer  who  is  removed  from  the  situation  at  hand 

Once  the  user  enters  the  number  of  mines,  that  number  is  laid  in  each  of  the  four 
depth  zones,  so  that  the  total  number  of  mines  in  the  channel  is  actually  four  times  the 
number  he  entered.  However,  since  the  sensor’s  performance  in  each  depth  zone  is 
assessed  separately,  the  effective  number  of  mines  in  the  search  is  the  number  entered  by 
the  user. 

The  X  and  y  coordinates  for  each  mine’s  position  are  determined  by  drawing 
uniform  random  numbers  that  are  bounded  by  the  dimensions  of  the  width  and  length  of 
the  channel,  respectively.  The  depth  dimension  is  fixed,  as  a  mine’s  relative  depth  within 
its  depth  zone  is  irrelevant  to  its  vulnerability  to  detection.  The  mines  remain  stationary. 

The  size  of  the  mines  is  not  addressed  in  ML-Recon.  When  the  tests  were  run  on 
the  system,  the  targets  used  were  generally  the  same  size  or  smaller  than  the  typical  moored 
mine.  The  data  upon  which  the  detection  capability  is  modeled  includes  detection/no 
detection  information  only,  since  the  sensor  itself  contains  the  classification  algorithm  for 
determining  whether  a  contact  is  a  mine  or  not. 

2.  The  Helicopter  Module 

Although  mine-hunting  plans  generally  consist  of  a  set  of  straight,  uniformly- 
spaced  tracks  flown  parallel  to  the  length  axis  of  the  channel,  there  are  a  number  of  physical 
influences  that  prevent  the  helicopter  pilot  from  flying  the  exact  plan.  If  there  were  no  error 
in  the  flight  path  of  the  helicopter,  it  would  be  flown  over  100%  of  the  channel  area  by 
choosing  the  number  of  tracks  as: 

Channel  Width 

Number  of  Tracks  =  - — —  (2) 

Sensor  Sweep  Wdth 

and  spacing  the  tracks  uniformly  across  the  width  of  the  channel,  offsetting  the  first  track 
by  one  half  of  a  track  width.  However,  wind  gusts,  inaccuracies  in  the  tracking  system 
onboard  the  helicopter,  and  pilot  error  all  cause  the  helicopter  to  deviate  from  the  intended 
track.  The  significance  of  this  navigational  error  is  in  the  resulting  holidays,  or  holes,  in 
the  search  plan.  Figure  6  is  a  theoretical  representation  of  holidays  caused  by  navigational 
error  in  a  five-track  search  plan.  Although  the  error  may  seem  exaggerated  for  the  flight 
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Holidays 


Figure  6.  Holidays  in  a  Search  Plan 


of  a  helicopter  with  a  precise  search  plan,  the  sample  of  actual  tracks  flown  during  the 
operational  tests,  shown  in  Appendix  A,  indicates  that  such  deviations  are  typical.  [It 
should  be  noted  that  data  such  as  the  sample  in  Appendix  A  is  only  available  through  a 
position-plotting  program  written  separately  from  the  Magic  Lantern  system  by  an  analyst 
at  the  firm  that  collected  the  data.  Such  information  is  not  known  to  the  typical  Magic 
Lantern  planner  before  or  after  the  search.] 

Figure  7  shows  a  frequency  histogram  of  the  actual  position  error  of  the  helicopter 
from  over  5000  data  points.  As  expected,  the  error  closely  resembles  a  normal  probability 
distribution.  The  navigational  error  data  for  the  helicopter  were  collected  once  per  second 
imder  various  wind  conditions.  Wind  speeds,  however,  were  recorded  only  once  per 
fifteen  minute  interval.  At  first  the  data  were  separated  into  high-wind  and  low -wind 
categories.  The  fact  that  there  was  not  any  difference  in  navigational  error  between  the  two 
categories  may  be  attributable  to  the  relatively  large  time  interval  between  wind  speed 
measurements.  Since  the  standard  deviation  under  both  conditions  was  the  same,  no 
influence  on  the  helicopter’s  cross-track  error  in  ML-Recon  is  attributed  to  wind  speed. 
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Figure  7.  Histogram  of  Helicopter  Navigational  Error 

To  represent  navigational  error  in  the  helicopter’s  flight  path,  the  area  searched  in 
the  model  is  first  presented  in  x-y  coordinates.  The  helicopter  flies  parallel  to  the  y-axis, 
and  its  cross-track  position  is  measured  on  the  x-axis.  The  centerline  for  each  planned 
search  track  (an  x  value)  is  calculated  before  the  flight  begins.  As  in  real  life,  the  ML- 
Recon  helicopter  is  intended  to  stay  directly  on  track  for  its  entire  length.  However,  to 
account  for  the  factors  mentioned  earlier,  the  helicopter  strays  randomly  to  either  side  of  the 
centerline.  If  this  error  were  simply  applied  by  adding  a  normal  random  number  to  the 
helicopter’s  lateral  position  at  one-second  intervals  (the  time  between  data  points  collected), 
the  flight  path  would  be  very  jagged  and  could  tend  to  drift  to  unrealistically  large  distances 
from  the  centerline  [Ref.  8].  By  applying  the  smoothing  process  described  below,  a  more 
realistic,  gradually  adjusting  flight  path  is  achieved. 

Suppose  the  helicopter’s  lateral  position  error  at  any  given  time,  t,  is  denoted  by  xt. 
At  regular  time  intervals  A,  a  new  position  with  x  and  y  components  is  assigned  to  the 
helicopter.  The  new  y  position  is  determined  simply  by  multiplying  A  by  the  helicopter’s 
speed  and  adding  it  to  the  previous  y  value.  For  the  new  cross-track  error  from  centerline, 
xt+A  ,  first  the  magmtude  of  the  previous  error,  xt,  is  damped  by  a  constant,  c,  where  : 
0  <  c  <  1.  This  keeps  the  helicopter  close  to  the  track,  yet  does  not  bring  it  all  the  way 
back  to  centerline.  In  order  to  determine  the  value  of  c,  another  constant,  x,  must  be 
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extracted  from  the  data  collected  during  the  Magic  Lantern  tests.  Described  by  Washburn 
as  a  relaxation  time  [Ref.  8],  t  is  calculated  using  the  equation: 

c  =  corr(x„  (3) 

The  correlation  coefficient  for  A  =  10  seconds  was  determined  to  be  0.77  from  the 
test  data.  This  describes  the  correlation  between  lateral  position  errors  at  ten-second 
intervals.  By  selecting  a  ten-second  interval,  sequential  lateral  position  error  measurements 
are  close  enough  to  be  dependent  on  one  another,  but  distant  enough  to  allow  the 
simulation  to  run  relatively  quickly.  A  shorter  interval  would  require  more  frequent  error 
assignments  in  the  program  and,  hence,  more  computer  processing  time.  Solving  for  r 

X  =  -A/  ln(0.77)  =  -10  sec/  -0.26  =  38.25  seconds  (4) 

Once  T  is  known,  the  damping  constant,  c,  is  calculated  using  equation  (3).  The  time 
between  instances  of  determining  a  new  lateral  position  error  is  referred  to  as  the 
navigational  time  interval.  Since  the  navigational  time  interval  for  ML-Recon  is  also  chosen 
to  be  ten  seconds,  the  damping  constant  is  identical  to  the  correlation  coefficient. 

After  the  previous  lateral  error  is  damped,  a  random  number  is  drawn  and  added  to 
it.  The  random  number  follows  a  normal  distribution  with  parameters  N(0,  q).  The 
variance,  q,  is  determined  using  the  data  from  the  tests  and  the  damping  factor,  c : 

q  =  -  c^s^  =  s^(l  -  c^)  (5) 

where  s  is  the  standard  deviation  of  the  error  data.  Summarizing  the  method  of  assigning 
each  new  lateral  position  error  to  the  helicopter. 

Xt+A  =  cx,  +  W,  (6) 

where  W  ~  N(0,  q).  The  graph  in  Figure  8  illustrates  the  process  by  which  the  helicopter 
is  assigned  its  lateral  position  error.  Since  the  helicopter  is  assigned  a  new  position  only 
every  ten  seconds,  it  flies  a  straight  line  from  one  assigned  position  to  the  next.  The 
sensor  emits  laser  pulses  every  0.375  seconds  and  requires  an  accurate  measurement  of  the 
helicopter’s  position  for  each  pulse.  Therefore,  a  simple  interpolation  of  the  helicopter’s 
position  along  the  straight  line  between  assigned  positions  is  calculated  when  prompted  by 
the  sensor  module. 
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This  process  of  inducing  a  realistic  time  correlation  structure  to  smooth  the 
helicopter’s  flight  path  is  known  as  the  Omstein-Uhlenbeck  (0-U)  process  [Ref.  8]. 
Current  MCM  planmng  tools  already  consider  the  standard  deviation  of  navigational  error 
[Ref.  6],  but  essentially  with  the  relaxation  time,  x,  equal  to  infinity.  By  analyzing  the 
error  data  and  modeling  it  in  ML-Recon  as  an  0-U  process,  representation  of  the  search 
with  a  realistic  relaxation  time  is  achieved. 

3.  The  Sensor  Module 

The  sensor  module  design  is  based  on  data  collected  during  operational  tests  of  the 
Magic  Lantern  system,  but  ML-Recon  does  not  follow  the  exact  steps  carried  out  by  the 
sensor.  For  instance,  ML-Recon  does  not  model  the  multiple  laser  pulses  emitted  by  the 
sensor  from  one  edge  of  the  sweep  width  to  the  other.  Such  detail  would  burden  the 
processor  and  result  in  extremely  time-consuming  operation  of  the  program  while  adding 
nothing  to  the  analysis.  Rather,  the  sweep  of  the  sensor  through  one  cycle  (as  described  in 
Chapter  11)  is  considered  as  one  event,  which  identifies  an  area  with  fixed  length  and 
width,  referred  to  as  the  \2isefs  footprint.  The  length  of  the  footprint  is  based  on  the  speed 
of  the  helicopter  and  the  frequency  of  the  pulses.  More  specifically,  successive  sweeps  are 
modeled  so  that  the  leading  edge  of  the  previous  footprint  is  adjacent  to  the  trailing  edge  of 
the  current  one,  given  that  the  helicopter  maintains  a  constant  speed.  Therefore,  there  is  no 
overlap  between  footprints  of  the  same  track.  The  width  of  the  footprint  is  based  on  the 
cumulative  width  of  a  cycle  of  pulses. 
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The  time  between  sweeps  in  ML-Recon  (0.375  seconds)  reflects  the  frequency  of 
the  actual  laser,  but  takes  into  account  the  lateral  consolidation  of  the  laser  pulses  into  one 
footprint  Since  there  is  no  overlap  from  one  footprint  to  the  next  in  the  same  track,  a  mine 
can  only  be  illuminated  by  the  sensor  once  per  track. 

As  the  helicopter  travels  along  each  track,  the  discrete,  independent  steps  of  ML- 
Recon’s  sensor  are  described  as  follows: 

•  Every  0.375  seconds,  the  sensor  emits  one  large,  rectangular  pulse  that 
penetrates  from  the  water’s  surface  through  all  four  depth  zones.  The  x-y 
coordinates  for  the  four  comers  of  this  pulse  are  precisely  calculated  using  the 
helicopter’s  location  at  that  instant  as  the  center  point  of  the  rectangle.  The  area 
inside  the  four  comers  is  referred  to  as  the  footprint. 

•  The  exact  position  of  each  randomly  laid  mine  is  then  checked  to  see  if  its 
position  is  within  the  footprint.  If  the  footprint  contains  no  mines,  no  further 
action  occurs  until  the  next  0.375  seconds  have  passed.  At  that  time  it  emits  a 
new  pulse,  identifying  a  new  footprint  (step  1 .). 

•  If  a  mine  is  determined  to  be  within  the  footprint,  its  depth  is  checked,  and  its 
lateral  range  from  the  center  of  the  footprint  is  calculated.  With  depth  and  lateral 
range,  the  appropriate  probability  of  detection  value,  can  then  be  applied. 

•  A  Uniform[0, 1]  random  number  is  generated  and  compared  to  using  the 

following  sequence: 

IFPj(ij)>  Random  Number  ~U[0,1] 

THEN  Detected  =  TRUE; 

ELSE 

Detected  =  FALSE; 

END 

At  the  end  of  each  replication,  the  total  number  of  mines  detected  in  each  depth  zone 
is  displayed  to  the  user  as  well  as  the  number  of  mines  that  were  missed.  When  all 
replications  have  been  completed,  a  separate  graph  is  presented  for  each  depth  zone, 
showing  performance  statistics  for  the  search.  Figure  9  is  an  example  of  a  typical  graph 
presented  for  one  depth  zone.  The  graph  shows  the  number  of  mines  detected  in  each 
replication  of  the  search  plan  as  well  as  the  mnning  average  of  mines  detected,  recalculated 
after  each  replication. 
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Search  Plan  Performance  Against  10  Mines 
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Figure  9.  Performance  Statistics 


After  the  four  performance  graphs  are  displayed,  the  user  is  given  the  choice  of 
viewing  an  animated  iteration  of  his  search  plan,  selecting  a  different  number  of  tracks  to  be 
replicated,  or  teiminating  the  program. 

D.  ANIMATION 

The  animation  of  the  search  plan  is  an  opportunity  for  the  planner  to  assess  whether 
the  number  of  tracks  selected  in  his  plan  is  sufficient  (see  Figure  6).  The  performance 
statistics  that  he  has  received  at  this  point  give  him  the  final  results,  but  they  do  not 
distinguish  between  mines  missed  because  of  poor  sensor  performance  and  mines  missed 
because  of  holidays  in  the  search  plan.  Analytical  planning  models  do  not  make  this 
distinction  either  since  they  generally  provide  expected  coverage  percentage,  but  do  not 
address  the  likelihood  that  a  mine  is  located  within  a  holiday. 

A  mine  that  is  located  inside  of  a  holiday  cannot  be  detected  because  it  is  never 
within  the  search  volume  of  the  sensor.  Thorough  searches  generally  do  not  include 
holidays  because  the  tracks  are  numerous  enough  to  make  up  for  navigational  errors. 
However,  if  the  planner  is  under  time  constraints,  he  may  not  have  the  luxury  of  saturating 
the  channel  with  additional  tracks.  In  these  instances,  an  animated  sample  of  his  plan  may 
convince  him  to  adjust  the  size  of  the  channel  or  request  additional  assets. 
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The  user  is  reminded  that  the  animation  is  simply  one  iteration  of  the  search  plan, 
and  that  it  does  not  represent  an  exact  prediction  for  the  performance  of  the  search. 
Nevertheless,  by  running  the  animation,  if  he  sees  large  and  numerous  holidays,  he  should 
be  persuaded  to  add  tracks  for  better  coverage. 

E.  KEY  MODEL  ASSUMPTION 

The  primary  assumption  in  ML-Recon  is  that  each  attempt  to  detect  a  mine  is 
independent.  There  is  no  environmental  phenomenon  or  shortcoming  in  the  sensor  that 
makes  some  mines  harder  to  detect.  As  long  as  a  mine  is  flown  over  by  the  helicopter  and 
is  within  the  depth  capabilities  of  the  sensor,  it  has  the  potential  to  be  detected. 
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IV.  THE  EXPERIMENT 


A.  OVERVIEW 

A  verification  test  for  any  simulation  model  is  performed  to  show  that  the  program 
behaves  as  intended,  checking  that  the  conceptual  model  (e.g.  flowcharts  and  assumptions) 
is  translated  into  a  correctly  working  computer  program.  Validation  of  a  simulation  is 
performed  by  showing  that  the  conceptual  model  is  accurate  [Ref.  9]. 

Verification  for  ML-Recon  is  carried  out  by  comparing  it  to  an  accredited  model 
currently  used  in  the  fleet  by  MCM  planners.  Further  analysis  is  conducted  on  ML-Recon 
alone.  This  is  done  to  explain  the  particular  features  of  the  simulation  that  make  it 
preferable  to  an  analytical  model  as  a  planning  aid. 

In  order  to  validate  ML-Recon,  it  must  be  compared  to  results  of  field  tests  separate 
from  those  used  in  building  the  model.  This  is  left  as  future  work  as  data  for  such  tests  are 
not  available  at  the  submission  of  this  thesis. 

B.  ML-RECON  VS.  UCPLN 

The  Uniform  Coverage  Plan  (UCPLN)  is  an  interactive  computer  program  currently 
used  by  MCM  planners  in  the  fleet  [Ref.  6].  The  model  uses  analytical  methods  to 
calculate  percent  clearance  achievable  in  a  potentially  mined  channel.  Inputs  to  UCPLN 
that  are  also  required  in  ML-Recon  are; 

•  area  length  and  width; 

•  navigational  error; 

•  percent  clearance  desired; 

•  speed 

•  sensor  sweep  width,  and; 

To  compare  ML-Recon  to  UCPLN,  some  modifications  to  the  code  are  required. 
For  sensor  sweep  width  ML-Recon  uses  a  lateral  range  function  with  five  different  Pj 
values  in  each  of  four  depth  zones.  The  UCPLN  program  uses  a  single  value  for  P^  across 
the  entire  sweep  width  of  the  sensor.  Therefore,  for  the  experiment,  the  P^  across  the 
lateral  range  is  aggregated  in  ML-Recon  to  one  value  for  each  depth  zone.  Rather  than 
further  aggregate  the  P^,  each  depth  zone  is  run  separately. 

The  test  scenarios  are  search  plans  consisting  of  5  through  16,  32,  48  and  64 
tracks.  Channel-size  is  kept  constant  throughout  the  experiment,  and  chosen  so  that  five 
tracks  would  provide  poor  coverage,  and  64  tracks  would  provide  nearly  complete 
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coverage.  One  hundred  replications  of  ML-Recon  were  run  for  each  search  plan.  Results 
are  compared  to  UCPLN  values  achieved  with  the  same  input  parameters . 
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V.  EXPERIMENTAL  RESULTS 


A.  OVERVIEW 

The  analysis  of  the  first  experiment  is  presented  with  a  higher  level  of  detail  since 
the  values  used  for  of  the  sensor  and  for  the  dimensions  of  the  laser  footprint  are 
notional.  The  subsequent  analysis,  using  the  real  values,  is  presented  in  a  way  that  is  clear 
after  the  notional  case  is  explained,  but  which  does  not  reveal  any  classified  information. 
The  final  analysis  is  done  on  ML-Recon  alone,  using  notional  numbers  to  describe  how  to 
attain  the  most  useful  results  when  planning  a  search. 

B.  ML-RECON  VS.  UCPLN  (Notional  Values) 

In  addition  to  the  similarity  in  results  between  ML-Recon  and  UCPLN,  the 
stochastic  nature  of  ML-Recon  is  evident  against  the  analytical  results  of  UCPLN  in  Figure 
10.  The  graph  is  taken  from  one  depth  zone  with  a  notional  P^  of  0.5. 


Simulation  (P^  =  0.5)  vs.  UCPLN 


Figure  10.  ML-Recon  Results  vs.  UCPLN 

The  following  analysis  is  conducted  in  order  to  determine  whether  or  not  the 
average  performance  generated  by  100  replications  of  ML-Recon  is  equivalent  to  the 
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analytically  derived,  predicted  performance  from  UCPLN  for  the  same  search  plan.  In  the 
analysis  the  following  symbols  are  used: 


I  -  depth  zone  index;  i  =  1,  2,  3,  4; 
n  -  number  of  replications  requested  by  the  user; 
r  -  replication  number  index;  r=  1,  2,  «; 

^wcPLN  ~  analytically  derived  percentage  of  mines  detected  with  UCPLN  in  depth  zone  i; 
x-j,  Xf2,’-.,  x.^-  randomsampleof  sizer  of  the  percentage  of  mines  detected  in 
ML-Recon  in  depth  zone  i,  with  mean  ju,.  and  variance  of. 

The  average  performance  of  the  simulation  for  any  particular  depth  zone,  i,  over  n 
replications  is  represented  by  the  random  variable,  3^ ,  where 

^  r-1 

Using  %  as  a  large-sample  estimator  for  x^^jcpln,  it  is  assumed  that  has  a  distribution 

that  is  approximately  normal  with  mean  ^  =  fi.  and  variance  of/Vn  [Ref.  10].  Since  the 

UCPLN  program  is  accredited  and  in  use  in  the  fleet,  x .  ^zjv  is  the  value  against  which 
ML-Recon’s  average  performance  is  tested.  The  null  and  alternate  hypotheses  for  this  test 
are  then: 


Hq  .  X,.  — 


against  ^ivcpw 


The  test  statistic,  Zj,  is  determined  by 


Vs 


The  actual  variance,  of,  of  x^^  is  not  known  but  can  be  estimated  (because  n  =  100,  used 
in  the  test,  is  sufficiently  large  [Ref.  10])  by  the  observed  variance,  sf,  where 


5f 
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The  observed  value  of  the  test  statistic  for  each  search  tested  is  given  for  each  depth,  i,  by 


Xi,VCPLN 


(10) 


Table  1  shows  the  values  achieved  by  ML-Recon  and  UCPLN  for  a  depth  zone  with  an 
aggregate  (notional)  of  0.5.  It  also  shows  the  difference  between  the  two  and  the  z  test 
statistic  for  each  search  plan.  Acceptance  or  rejection  of  the  null  hypothesis  is  based  on  a 

two-tailed  z-test  using  an  a  of  0.05.  As  the  table  shows,  only  two  rejections  were 
observed  out  of  the  15  comparisons  conducted  in  this  experiment. 


ML“Recon 

Difference 

ML-R  STDEV 

z  stat 

Acceot/Reiect 

5 

11.73 

12.68 

0.95 

6.41 

1.48 

Accept 

6 

13.9 

15.2 

1.3 

7.8 

1.67 

Accept 

7 

16.44 

16.76 

0.32 

6.84 

0.47 

Accept 

8 

18.9 

20.6 

1.7 

8.8 

1.93 

Accept 

9 

20.51 

21.28 

0.77 

8.22 

0.94 

Accept 

10 

23.61 

25 

1.39 

8.85 

1.57 

Accept 

1  1 

26.36 

28.48 

2,12 

8.66 

2.45 

Reject 

12 

28.38 

29.32 

0.94 

9.45 

0.99 

Accept 

13 

30.34 

30.88 

0.54 

9.34 

0.58 

Accept 

14 

32.25 

31.84 

-0.41 

8.64 

-0.47 

Accept 

15 

34.1 1 

34.4 

0.29 

9,27 

0.31 

Accept 

16 

35.91 

36.44 

0.53 

7.34 

0.72 

Accept 

32 

58.9 

58.88 

-0.02 

8.61 

-0.02 

Accept 

48 

73.62 

69.48 

-4.14 

8.89 

-4.66 

Reject 

64 

83.04 

82.88 

-0.16 

7.67 

-0.21 

Accept 

Table  1.  ML-Recon  vs.  UCPLN  Results  (Notional  Values). 


C.  ML-RECON  VS.  UCPLN  (Actual  Values) 

The  same  two-tailed  z-test  was  conducted  to  compare  ML-Recon ’s  average  results 
to  UCPLN  using  the  P^  values  determined  in  the  operational  tests  of  Magic  Lantern.  The 
results  shown  in  Table  2  are  from  a  comparison  of  the  third  depth  zone  (Z3)  to  the 
UCPLN-generated  value  for  the  number  of  mines  detected  in  each  search  plan.  Although 
the  actual  performance  of  ML-Recon  is  omitted  for  classification  reasons,  the  other 
statistics,  as  described  in  the  notional  case,  are  included.  Results  for  the  other  three  depth 
zones  are  similarly  presented  in  Appendix  B.  As  in  the  notional  case,  the  number  of 
rejections  at  each  depth  zone  is  two,  although  they  do  not  occur  for  the  same  number  of 
tracks  in  each  case. 
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Tracks 

UCPLN  -  MLRecon 

ML-R  STDEV 

z  statistic 

Acceot/Relect 

5 

-0.22 

9.97 

-0.22 

Accept 

6 

0.62 

10.19 

0.61 

Accept 

7 

0.21 

8.75 

0.24 

Accept 

8 

1.22 

11.55 

1.06 

Accept 

9 

1.64 

10.26 

1.60 

Accept 

10 

-1.81 

9.19 

-1.97 

Reject 

n 

-0.58 

9.09 

-0.64 

Accept 

12 

-0.22 

10.81 

-0.20 

Accept 

13 

-0.13 

8.98 

-0.14 

Accept 

14 

-1.53 

9.09 

-1.68 

Accept 

15 

-0.78 

7.45 

-1.05 

Accept 

16 

0.46 

8.58 

0.54 

Accept 

32 

0.88 

3.90 

2.26 

Reject 

48 

0.04 

1.82 

0.22 

Accent 

64 

-0.07 

0.95 

-0.73 

Accept 

Table  2.  ML-Recon  vs.  UCPLN  (Actual  Values). 


D.  MINE  DENSITY’S  AFFECT  ON  PERFORMANCE  VARIABILITY 

The  Monte  Carlo  approach  to  formulating  a  sea  mine  reconnaissance  plan  provides 
the  user  with  more  information  than  just  the  expected  average  performance  of  the  Magic 
Lantern  system.  The  requirement  for  the  user  to  enter  the  number  of  mines  in  the  channel 
was  mentioned  in  the  description  of  the  model.  Figure  9  shows  an  example  of  a  typical 
performance  graph  displayed  to  the  user  when  the  replications  have  been  completed.  In 
that  graph,  as  the  number  of  replications  increases,  the  running  average  becomes  more  flat. 
This  indicates  that  the  best  estimate  for  the  average  number  of  mines  detected  with  that 
particular  search  plan  in  that  depth  zone  is  achieved  by  running  as  many  replications  as 
practical.  The  individual  results  of  each  search  on  the  graph  shows  the  planner  how  much 
worse  than  the  average  his  plan  may  perform.  This  variation  in  results  depends  on  the 
number  of  mines  entered  by  the  user  at  the  beginning  of  the  program. 
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The  reason  for  this  direct  relationship  between  variance  and  the  number  of  mines  in 
the  channel  is  that  the  number  of  mines  detected  in  each  depth  zone  on  any  replication,  for 
the  purposes  of  this  analysis,  is  approximated  by  a  binomial  distribution.  This  is  illustrated 
by  equating  ML-Recon  to  the  definition  of  a  binomial  experiment  [Ref.  1 0] . 

•  The  experiment  consists  of  m  identical  trials,  or  detectable  mines  in  any 
particular  depth  zone. 

•  Each  trial  results  in  success  or  failure  with  probability  p ,  or,  each  mine  is 
detected  with  probability  P ^  in  each  depth  zone  and  lateral  range. 

•  The  trials  are  assumed  to  be  independent,  as  each  mine  that  is  flown  over  has  the 
same  chance  of  being  detected  as  any  other  mine,  regardless  of  whether  others 
have  been  detected  or  not.* 

•  The  random  variable,  x,.,  represents  the  number  of  mines  detected  out  of 
the  m  total  mines  in  any  given  depth  zone,  i. 

Having  identified  the  distribution  of  the  number  of  mines  detected  in  any  depth  zone 
as  approximately  binomial,  the  relationship  between  the  number  of  mines  in  the  channel  to 
begin  with  and  the  variance  in  the  number  detected  can  be  explained  using  the  following 
notation: 

m  =  number  of  mines  in  each  depth  zone; 

Pi  -  aggregate  probability  of  detecting  any  mine  in  depth  zone  i; 

X,.  =  no.  of  mines  detected  in  depth  zone  i  for  a  given  search  plan;  x,.  ~  Bin(i»,  />,.); 
Since  x.  is  assumed  to  follow  a  binomial  distribution. 


E[xJ  s  mpi 

(11) 

E[x/i«]  =p. 

(12) 

Here  it  is  clear  that,  by  increasing  the  number  of  mines  in  the  channel,  the  expected  fraction 
of  mines  detected  does  not  change,  since/?,,  is  constant.  Therefore,  the  number  of  mines 
does  not  affect  the  average  performance  of  the  search  plan.  However,  the  variance 
equation  for  the  fraction  of  mines  detected  shows  that,  as  m  increases,  the  variance  of  the 
fraction  of  mines  detected  decreases. 

Var[x/jii]  =  l/m^  Var[x,.]  =  pll-p^m  (13) 


*  It  should  be  noted  that,  in  reality,  there  is  a  very  slight  dependence  between  trials.  Since  there  is  a 
random  cross-track  in  error  in  the  helicopter’s  flight  path,  consider  the  unlikely  casein  which  every  track  of 
the  plan  is  actually  flown  outside  of  the  channel.  None  of  the  mines  would  have  a  chance  of  being  detected, 
they  all  are  share  the  same  flawed  search  path,  and  their  chances  of  being  detected  are,  therefore,  not 
independent.  However,  since  such  a  case,  or  any  one  like  it,  is  so  unlikely,  the  independence  assumption  is 
made. 
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Therefore,  by  increasing  the  number  of  mines  in  the  channel,  the  spread  in  the  results  is 
reduced,  although  the  average  remains  the  same. 

Figure  1 1  gives  a  visual  sample  of  this  phenomenon.  In  both  of  the  graphs  shown, 
40  tracks  were  run  for  30  replications,  and  the  sensor’s  probability  of  detecting  a  mine  was 
0.5.  The  size  of  the  channel  in  each  case  was  the  same.  The  difference  in  the  two  is  that 
the  one  on  the  left  is  for  a  channel  with  10  mines,  and  the  one  on  the  right  is  for  a 
channel  with  100  mines.  The  average  fraction  of  mines  detectedin  each  plan,  jc/#w,  is 


Figure  11.  Results  From  Channel  With  10  Mines  and  Channel  With  100  Mines. 


approximately  equal  (i.e.  8.8/10  «  87.8/100).  However,  the  standard  deviation  of  x/m 
is  substantially  smaller  in  the  graph  on  the  right: 

JL64  <  0.92  (14) 

100  10 

This  is  attributable  to  the  binomial  nature  of  x,.. 

The  possibility  also  exists  that  the  planner  is  interested  in  the  actual  number  of 
mines  remaining  undetected  in  the  chaimel  after  the  search  is  complete,  regardless  of  the 
fraction  of  mines  detected.  ML-Recon  accommodates  this  approach  by  printing  the  number 
of  undetected  mines  after  each  replication. 
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VI.  CONCLUSION 


The  simulation  presented  in  this  thesis,  ML-Recon,  meets  the  MCM  planner’s 
requirements  for  a  planning  tool  that  accurately  represents  a  conceptual  model  of  the  Magic 
Lantern  system.  It  provides  him  with  the  information  that  is  essential  to  formulating  an 
efficient  and  thorough  search  plan. 

ML-Recon’s  stochastic  treatment  of  the  helicopter’s  cross-track  error  is  a  unique 
feature  that  gives  the  planner  insight  toward  the  likelihood  of  holidays  in  his  plan.  The 
stochastic  representation  of  the  sensor  also  contributes  to  a  spread  of  realistic  results  based 
on  many  replications.  By  using  this  model  prior  to  deploying  the  system,  he  can  be 
confident  that  his  plan  is  thorough  enough  that  all  potential  mines  will  at  least  be  flown 
over,  yet  not  so  ambitious  that  it  will  have  to  be  terminated  prematurely  to  refuel  the 
helicopter. 

Should  there  be  improvements  to  the  Magic  Lantern  system,  or  if  it  is  adapted  to  a 
different  helicopter,  the  model  can  continue  to  meet  the  planner’s  needs  by  simply  replacing 
the  applicable  descriptive  parameters. 
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APPENDIX  A.  MAGIC  LANTERN  HELICOPTER  PLOT 


The  figure  below  is  a  record  of  a  helicopter’s  actual  tracks  during  Magic  Lantern 
field  tests  conducted  in  July,  1994.  The  units  for  the  x  and  y  axes  are  omitted  for  security 
reasons.  The  intent  of  this  graphic  is  to  illustrate  the  error  in  the  helicopter’s  flight  path. 
The  search  plan  for  this  sortie  was  a  set  of  straight,  uniformly-spaced  tracks. 
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APPENDIX  B.  ML-RECON  COMPARED  TO  UCPLN 


Tables.  ML-Recon  vs.  UCPLN  (Actueil  Values)  For  First  Depth  Zone  (Zl). 


Table  4.  ML-Recon  vs.  UCPLN  (Actual  Values)  For  Second  Depth  Zone  (Z2). 
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T  racks 

UCPLN-MLRecon 

ML-R  STDEV 

z  stat 

Acceot/Reiect 

5 

0.26 

8.74 

0.30 

Accept 

6 

-0.18 

10.78 

-0.17 

Accept 

7 

0.69 

9.56 

0.72 

Accept 

8 

-0.22 

9.55 

-0.23 

9 

-0.23 

8.83 

-0.26 

10 

2.91 

13.74 

2.12 

1 1 

-2.07 

9.53 

-2.17 

Reject 

12 

-0.76 

8.05 

-0.94 

Accept 

13 

1.55 

8.99 

1.72 

Accept 

14 

-0.75 

8.58 

-0.87 

Accept 

15 

0.97 

9.07 

1.07 

Accept 

16 

-0.54 

8.99 

-0.60 

Accept 

32 

-0.03 

4.62 

-0.06 

Accept 

48 

0.24 

2.68 

0.90 

Accept 

64 

-0.1 

1.45 

-0.69 

Accept 

Table  5.  ML-Recon  vs.  UCPLN  (Actual  Values)  For  Fourth  Depth  Zone  (Z4). 
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